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M summary frames selected represent video content span- 
ning a longer period of time than mere the most recent M 
summary. An alternative embodiment is to also embed the 
summary of video of the future, i.e. about some time t, 2t, 
3t, . . . after the current playback. 5 

Another method of interleaving the selection of summary 
frames is shown in FIG. 14. Here, the Summary frames are 
listed row- wise in a triangular structure, i.e., each row has 
more columns than the previous one. At each step of the 
embedding, the frames are read off in a column fashion. For 10 
example, if each embedding step requires 4 frames, then fsl, 
fs2, fs4 and fs7 are first used. At the next step, fsl2, fcl9, fc3 
and fs5 are used. This provides a non-uniform sampling in 
the sense that more summary frames are used that are closer 
to the current frame, and fewer frames are used that are 35 
further from the current frame. This achieves what is pre- 
sented in FIG. 6B. 

In an interactive video playback environment, such as the 
desktop or video streaming, the summary frames displayed 2Q 
with the playback frames can be further used for byperlink- 
ing purposes, i.e., a user could click (or select) frames f st and 
the playback will continue from frame f sim 

A preferred use of summary-hyperlinking is now 
described. At the playback of some current shot s it the 25 
summary frames of the past shots and future 1 shots are 
simultaneously shown. The user can then easily see and 
replay one of the recent 1 shots or skip to one of the next 
shots; this is in contrast to the current method of video 
playback, in which a user can only see the current frame 30 
being playback. 

Another preferred use of summary-hyperlinking is to 
embed the summary frames not of the current video being 
played back, but to embed the summary of video of another 35 
program. A user can then click (or choose) to jump to that 
particular. An example use would be showing the summary 
of another football game while the video playback is of 
another. A user can then watch the current game but also 
grasp the summary of the other game. He can also switch to 
the other game directly without the needing to know the 
channel. 

For digital video compressed with standards such as 
MPEG-1 and MPEG-2, a preferred method to embed the 45 
summary frames is at the user-data area, which are specified 
by a particular header. To further ensure that the summary 
frames from the bitstreams can be extracted and displayed 
from any point (i.e., not necessarily from the very 
beginning) in the bitstreams, a new set of summary frame is 
embedded at fixed intervals T For an MPEG-1 video (352 
by 240 pixels) at 1.5 Mbits/sec, a DC image takes up 1.8 
Kbytes. If in one sec, 4 DC images are embedded, then the 
overhead is 7.2/192 or 3.75%. If instead, 1 DC image is 55 
embedded in each second, the overhead is 0.09375 %. For 
an MPEG-2 video (704 by 480 pixels) at 4.0 Mbits/sec, a 
DC image takes up 7.92 Kbytes. The overhead for embed- 
ding 4 DC images/sec is 6.1% and for 1 DC image/sec is 
1.5%. For higher bit rate MPEG-2 such as 6.0 M bits/sec, 60 
the respective overhead becomes 4% and 0.38%. The 
embedding process operates directly on the compressed 
video and does not require any decompression, and thus very 
computationally efficient. 65 

While the invention has been described in terms of 
preferred embodiments, those skilled in the art will recog- 
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nize that the invention can be practiced with modification 
within the spirit and scope of the appended claims. 

What is claimed is: 

1. A video viewing system, comprising: 

a display screen for viewing a video program in progress; 
and 

at least one summary frame superimposed on said display 
screen along with said video program in progress, said 
at least one summary frame comprising a past frame 
from said video program in progress, 

wherein selecting said at least one summary frame hyper- 
links the video program in progress to begin playing 
from said selected summary frame. 

2. A video viewing system comprising; 

a display screen for viewing a video program in progress; 

at least one summary frame superimposed on said display 
screen along with said video program in progress, said 
at least one summary frame comprising a past frame 
from said video program in progress; and 

at least one preview frame comprising a future frame from 
said video program in process, 

wherein selecting one of said summary fame and said 
preview frame hyperlinks the video program in 
progress to begin playing from said selected one of said 
summary frame and said preview frame. 

3. A user interactive video viewing system comprising; 

a display screen for viewing a video program; 

at least one summary frame superimposed on said display 
screen said at least one summary fame comprising one 
of a past or future frame from said program; and 

a control means for allowing a user to change said video 
program and for allowing said user to select said at least 
one summary frame to play at least a segment of said 
video program corresponding to said selected summary 
frame, 

wherein said viewing system comprises client connected 

to a server, and 
wherein selecting said at least one said summary frame 

hyperlinks said server to start playing said video from 

said summary frame, 

4. A method for informing a viewer of the content of a 
video program in progress, comprising the steps of: 

selecting a plurality of summary frames depicting selected 
events from a video program; 

embedding said summary frames in said video program; 

transmitting said video program comprising said sum- 
mary frames over a media; 

displaying said video program comprising said summary 
frames on a screen; 

hyperlinking to a corresponding portion in said video 
program when a particular summary frame is selected 
by a viewer; and 

playing said video program from said corresponding 
portion in said video program. 

5. A method for informing a viewer of the content of a 
video program in progress, comprising the steps of: 

selecting a plurality of summary frames depicting selected 
events from a video program; 
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embedding said summary frames in said video program; 

transmitting said video program comprising said sum- 
mary frames over a media; and 

displaying said video program comprising said summary 
frames on a screen, 

wherein said selecting step comprises the steps of; 
scanning a first frame of said video and designating 

said first frame as a key frame; 
scanning next frame in said video; 1Q 
comparing said next frame to said keyframe with a 
difference function, 
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if a result of said difference function is negative scan- 
ning another frame in said video, 
if said result of said difference function is true, then 
designating said next frame as said key frame; 
comparing said key frame with summary frames 
currently being displayed to find one with a small- 
est difference; and 
replacing said one with a smallest difference with 
said key frame. 

***** 
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